Various forms of network-based storage systems are known today. These forms include network attached storage (NAS), storage area networks (SANs), and others. Network storage systems are commonly used for a variety of purposes, such as providing multiple users with access to shared data, backing up critical data (e.g., by data mirroring), etc.
A network-based storage system typically includes at least one storage server, which is a processing system configured to store and retrieve data on behalf of one or more client processing systems (“clients”). In the context of NAS, a storage server may be a file server, which is sometimes called a “filer”. A filer operates on behalf of one or more clients to store and manage shared files. The files may be stored in a storage subsystem that includes one or more arrays of mass storage devices, such as magnetic or optical disks or tapes, by using RAID (Redundant Array of Inexpensive Disks). Hence, the mass storage devices in each array may be organized into one or more separate RAID groups.
In a SAN context, a storage server provides clients with block-level access to stored data, rather than file-level access. Some storage servers are capable of providing clients with both file-level access and block-level access, such as certain Filers made by Network Appliance, Inc. (NetApp®) of Sunnyvale, Calif.
In file servers, data is stored in logical containers called volumes, which may be identical with, or proper subsets of, aggregates. An “aggregate” is a logical container for a pool of storage, combining one or more physical mass storage devices (e.g., disks) or parts thereof into a single logical storage object, which contains or provides storage for one or more other logical data sets at a higher level of abstraction (e.g., volumes). A “volume” is a set of stored data associated with a collection of mass storage devices, such as disks, which obtains its storage from (i.e., is contained within, and may be coextensive with) an aggregate, and which is managed as an independent administrative unit, such as a complete file system. A “file system” is an independently managed, self-contained, hierarchal set of data units (e.g., files, blocks or LUNs). Although a volume or file system (as those terms are used herein) may store data in the form of files, that is not necessarily the case. That is, a volume or file system may store data in the form of other units, such as blocks or LUNs.
A storage server may maintain one or more write-out-of-place file systems. In a write-out-of-place file system, whenever a data block is modified, it is written to a new physical location on disk. This is in contrast with a write-in-place approach, where a data block, when modified, is written in its modified form back to the same physical location on disk. An example of file system software that implements write-out-of-place is the WAFL® file system software included in the Data ONTAP® storage operating system of NetApp.
One feature which is useful to have in a storage server is the ability to create a read-only, persistent, point-in-time image (RPPI) of a data set, such as a volume or a LUN, including its metadata. This capability allows the exact state of the data set to be restored from the RPPI in the event of, for example, data corruption or accidental data deletion. The ability to restore data from an RPPI provides administrators with a simple mechanism to revert the state of their data to a known previous point in time as captured by the RPPI. Typically, creation of an RPPI or restoration from an RPPI can be controlled from a client-side software tool. An example of an implementation of an RPPI is a Snapshot™ generated by SnapDrive™ or SnapManager® for Microsoft® Exchange software, both made by NetApp. Unlike other RPPI implementations, NetApp Snapshots do not require duplication of data blocks in the active file system, because a Snapshot can include pointers to data blocks in the active file system.
An example of an RPPI technique which does not require duplication of data blocks to create an RPPI is described in U.S. Pat. No. 5,819,292, which is incorporated herein by reference, and which is assigned to NetApp. The described technique of creating an RPPI (e.g., a Snapshot) does not require duplication of data blocks in the active file system, because the active file system can include pointers to data blocks in an RPPI, for any blocks that have not been modified since the RPPI was created. (The term “Snapshot” is used in this document without derogation of Network Appliance, Inc.'s trademark rights.) Among other advantages, this technique allows an RPPI to be created quickly, helps to reduce consumption of storage space due to RPPIs, and reduces the need to repeatedly update data block pointers as required in some prior art RPPI techniques.
Write out-of-place technology allows for efficient operation of a backup process through use of the RPPI technique discussed above. As shown in FIG. 1, a storage server 2 implemented with write out-of-place file systems may serve as a backup server. Backing up of a file system (i.e., a volume, LUN, etc.) or a subset thereof on a primary storage system 1 (hereinafter “primary server”) starts with the step of completely copying the data of the file system or subset on the primary server 1 to the backup server 2. This initial, or baseline, transfer may take some time to complete, as it is duplicating the entire source data set on the backup server 2 much like a full backup to tape. When the initial full backup is performed, the backup server 2 creates an RPPI of the volume just copied over. Each subsequent backup transfers only the data blocks which have changed since the previous backup. The backup server 2 receives these data blocks, updates its volume, and creates a new RPPI. This backup mechanism is called a block-level incremental backup process. An example of such a backup system is the SnapVault® system of NetApp.
A shortcoming with the above backup mechanism is that there can be too many RPPIs created on the backup server, because each time an update occurs on the backup server, the backup server needs to create an RPPI. However, the number of RPPIs a backup server can maintain is limited by design or physical considerations (e.g., available storage space). This shortcoming can be overcome by coordinating backups from all primary servers such that they occur at the same frequency on similar schedules. Then, a single RPPI can be taken to capture the changed data in all backups. This is the approach taken by NetApp's SnapVault system. This approach, however, requires the coordination and loses the flexibility of a backup system. Thus, another way is needed to reduce the number of RPPIs created on the backup server, yet still ensure that each and every update of a backup server is captured.